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FINAL  REPORT 


GRANT  #:  N00014-95-1-0440 

PRINCIPLE  INVESTIGATOR:  Ron  Sun 

INSTITUTION:  University  of  Missouri-  Columbia 

GRANT  TITLE:  Hybrid  models  for  sequential  decision  making 

AWARD  PERIOD:  January  1,  1995  -  May  31,  1999 

OBJECTIVE:  To  investigate  hybrid  architectures  of  complex  learning,  especially  as  applied 
to  a  simulated  minefield  navigation  task. 

APPROACH:  Both  human  experiments  and  model  developments  were  undertaken.  Model 
developments  were  informed  by  human  data,  which  in  turn  helped  to  design  further  human 
experiments. 

ACCOMPLISHMENTS:  We  developed  a  hybrid  connectionist  model  CLARION  as  a  demon¬ 
stration  of  the  approach  of  bottom-up  skill  learning  (from  procedural  to  declarative  knowl¬ 
edge).  The  model  essentially  consisted  of  two  levels  for  capturing  both  procedural  and 
declarative  knowledge  and  enabling  bottom-up  learning,  which  differed  markedly  from  ex¬ 
isting  models.  Our  experiments  during  the  past  five  years  demonstrated  matches  of  the 
model  with  human  data  in  several  domains.  Our  focus  has  been  on  the  minefield  navigation 
(MN)  task.  A  number  of  other  skill  learning  tasks  have  also  been  chosen  to  be  simulated 
that  span  the  spectrum  ranging  from  simple  reactive  skills  to  more  complex  cognitive  skills. 
The  tasks  include  serial  reaction  time  (SRT)  tasks,  dynamic  control  (DC)  tasks,  the  Tow¬ 
er  of  Hanoi  (TOH)  task.  A  good  fit  was  found  between  the  model  and  the  human  data, 
although  the  work  is  not  yet  complete. 

In  addition,  several  different  ways  of  incorporating  symbolic  structures  to  enhance  rein¬ 
forcement  learning  are  developed  in  some  detail.  Different  from  existing  work  using  symbolic 
structures  in  RL,  we  focus  on  autonomous  learning  from  scratch  without  requiring  a  priori 
domain-specific  knowledge.  Thus  the  role  of  symbolic  methods  lies  truly  in  enhancing  learn¬ 
ing,  not  in  providing  a  priori  domain-specific  knowledge  to  alleviate  the  need  for  learning. 
The  key  advantage  of  our  methods  is  that  they  are  more  autonomous,  more  generic,  and 
less  expensive  to  apply,  compared  with  existing  methods  that  rely  more  on  a  priori  knowl¬ 
edge.  Some  of  our  methods  generate  explicit,  symbolic  knowledge  that  are  more  suitable  for 
comprehension,  explanation,  and  communication  of  resulting  knowledge.  Several  different 
domains  and  their  variations  have  been  tackled  by  our  methods. 

CONCLUSIONS:  A  new  theoretical  framework  has  emerged  that  suggests  a  new  way  of 
understanding  skill  learning  —  bottom- up  learning  that  goes  from  procedural  to  declarative 
knowledge.  There  is  still  much  work  that  needs  to  be  done.  Such  work  include:  further 
development  of  techniques  of  autonomous  learning,  further  development  of  cognitive  models 
and  verifications  of  such  models  by  investigating  their  fit  with  new  cognitive  data  in  new 
cognitive  domains,  and  related  experimental  and  theoretical  work.  Such  work  can  lead  to 
significant  advances  in  several  areas. 

SIGNIFICANCE:  The  model  we  developed  suggests  the  possibility  of  alternative  ways  of 
skill  learning  different  from  existing  accounts,  and  highlights  an  approach  that  has  been 


1 


largely  neglected.  Comparisons  with  existing  learning  models  (in  AI  and  psychology)  showed 
that  our  approach  as  embodied  in  CLARION  has  some  unique  characteristics  that  other 
approaches  or  models  did  not  capture,  most  notably  the  bottom-up  development  and  con¬ 
tinuous  interaction  of  both  declarative  and  procedural  knowledge  (different  from  top-down 
declarative-to-procedural  learning,  and  beyond  separate,  parallel  learning  of  the  two  types 
of  knowledge).  The  match  with  human  data  across  various  conditions/manipulations  is 
significant,  in  that  it  provides  evidence  with  regard  to  the  cognitive  validity  of  the  model 
as  well  as  providing  new  ideas  for  further  research. 
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